fix: reset AV1 coding state on every key frame - #31
Conversation
609cd1e to
116ef31
Compare
|
jfyi: the actual fix is ~3 lines in record.rs - the rest of the diff is the rc_keyframes example, so that the issue can be showcased/reproduced without a full streaming setup (fails on main at 12.79 dB, passes with the fix at 67.60 dB). |
|
Putting into draft for now. For some reason started observing weird behaviour on AC4 and AV1 + HDR path again. There are no green screen artifacts anymore, but on transitioning from SDR to HDR after intro movies the game now exits fullscreen (on moonlight-qt) and just black-screens. Sound goes, stream is live, but client is unable to decode frames for whatever reason. Wasn't happening few days ago on moonshine-git, need to do a little more debugging there to claim this is the definitive fix. |
116ef31 to
e49b3a1
Compare
|
un-drafting - the black screen wasn't this fix. did a proper bitstream capture this time (dumped the encoded stream host-side at the SDR→HDR transition):
turns out the black screen was my test bundle's fault, unrelated to this PR - with this fix in I can't repro it anymore, with or without HDR metadata OBU injection on top (un-gating that injection is a separate fix, hgaiser/moonshine#166). rebased on main (v0.8.1), |
e49b3a1 to
9764c4c
Compare
NVIDIA's Vulkan AV1 encoder silently emits undecodable key frames mid-session when rate control is active: the tile data doesn't match the emitted frame header, dav1d/ffmpeg reject the frame with a hard parse error, and every frame until the next key frame is lost with it. Only the first key frame of a session was valid, so any stream using CBR/VBR with on-demand IDRs (game streaming) broke permanently on the first IDR request. CQP was unaffected, so the CQP-based examples never caught it. A key frame is a clean restart point, so re-issue the RESET + ENCODE_RATE_CONTROL + ENCODE_QUALITY_LEVEL control command there (same shape as the first-frame setup) instead of only on the first frame. In theory this is generic, spec-compliant behavior regardless of vendor, though tested only on NVIDIA. Scoped to active rate control: with rate control disabled there is no state worth resetting, so CQP skips it. The new rc_keyframes example guards the regression: CBR with a short GOP, then a full-stream ffmpeg decode + PSNR check. It fails on current main at 12.79 dB and passes with this fix at 67.60 dB on RTX 5060 Ti (driver 610.43.03); CQP behavior unchanged.
9764c4c to
6cc130b
Compare
|
You can never hate NVIDIA enough 😅 Changes look good from quick read, unfortunately got no RTX 2060 anymore to test with, since it was mostly sitting idle draining power, I sold it on the used-market. I'll verify with the RX 9060 XT when I get the chance 👍 |
2 participants
NVIDIA's Vulkan AV1 encoder silently emits undecodable key frames mid-session when rate control is active: the tile data doesn't match the emitted frame header, dav1d and ffmpeg reject the frame with a hard parse error, and every frame until the next key frame is lost with it. Only the first key frame of a session was valid, so any stream using CBR/VBR with on-demand IDRs (game streaming) broke permanently on the first IDR request - in practice a permanent greenscreen the moment the client asks for a recovery key frame (this is the "potential fix" I mentioned in hgaiser/moonshine#156). CQP was unaffected, so CQP-based examples never caught it.
The fix re-issues the coding-state RESET + rate control + quality level control command on every key frame instead of only the first, same shape as the existing first-frame setup. A key frame is a clean restart point, so in theory this is generic and spec-compliant behavior regardless of the vendor (though I tested only on Nvidia). Scoped to active rate control: with rate control disabled there's no state worth resetting, so CQP skips the extra control command entirely.
Verification - new
rc_keyframesexample following therfi.rspattern: AV1 + CBR +gop_size10 so key frames land mid-stream, then a full-stream ffmpeg decode + PSNR check against the source. On current main it fails at 12.79 dB (the stream decodes but is corrupt from the first mid-stream key frame on - the soft failure mode a plain decode-success check would miss); with this fix it passes at 67.60 dB. Also verified end to end in a real moonshine session (4K/120 AV1): pre-fix greenscreens on the first recovery key frame, post-fix recovers cleanly.Closes hgaiser/moonshine#155.
Host: NVidia 5060 Ti (driver 610.43.03), Arch